Bridging Long Time

Numerous recent papers (including many NIPS papers) focus on standard recurrent nets' inability to deal with long time lags between relevant input signals and teacher signals. Rather sophisticated, alternative methods were proposed. We rst show: problems used to promote certain algorithms in numerous previous papers can be solved more quickly by random weight guessing than by the proposed algorithms. This does not mean that guessing is a good algorithm. It just casts doubt on whether the other algorithms are, or whether the chosen problems are meaningful. We then use long short term memory (LSTM), our own recent algorithm, to solve hard problems that can neither be quickly solved by random weight guessing nor by any other recurrent net algorithm we are aware of.